So you've built your machine learning or deep learning model. That final stage – the crucial cog in your machine learning or deep learning project – is model deployment. You need to be able to get the model to the end user, right? And yet you'll face a ton of questions about model deployment when you sit for data scientist interviews! What are the different tools for model deployment?

Very first (and very draft) round of Machine Learning and Artificial Intelligence Frameworks and Applications Fingerprinting and Internet Census project. Based on the Grinder framework. Why? AI systems are complex and fragile. For example "The TensorFlow server is meant for internal communication only. It is not built for use in an untrusted network."

Working with TensorFlow Serving I thought, it would be really great to serve the Keras models too. The advantage of Keras is obvious -- it significantly simplifies the model development and allows trying of the models much faster than the pure TensorFlow framework. Another motivation, I wanted to make a client independent of the huge TensorFlow framework and use a very limited part of the Serving capabilities. Keras provides high-level neural network API and is capable to run on top of TensorFlow, CNTK or Theano. Basically, it abstracts those frameworks, is much easier to understand and learn and allows you to do more with less code.

A while ago I wrote about Machine Learning model deployment with TensorFlow Serving. The main advantage of that approach, in my opinion, is a performance (thanks to gRPC and Protobufs) and direct use of classes generated from Protobufs instead of manual creation of JSON objects. The client calls a server as they were parts of the same program. That makes the code easy to understand and maintain. Now we host our model somewhere (for instance here) and can talk to it over gRPC using our special client.